1 EDA

We can first visualize the data after dimension reduction with zinbWave. We then use t-SNE and color the cells with the allen labels. Since we use various values for the k parameter, we plot two representations with the most extreme values of k.

2 General comments

We have the following workflow

In the Seurat clustering, there are two parameters to choose from: the resolution and the k.param (use in a knn step). From the seurat help file, for the resolution parameter: use a value above (below) 1.0 if you want to obtain a larger (smaller) number of communities.

We picked a value of 1.6 as we want many clusters to start with. We also pick k = 50. Those values seem intermediate in term of ARI with other pairs of paramters (see section Seurat ARI param)

We also consider two ways of computing the ARI between RSEC and other methods. Either, we force RSEC to assign all cells to a given cluster, or we only compute the ARI between RSEC and the other methods on those cells that RSEC do cluster. Note that in the second case, other pairs of methods are compared using all cells. We denote as RsecT the cluster assignement where all cells are assigned (i.e Rsec Total).

The ARI merging method works as follow. We iterate over all pairs of clusters for every clustering method. For each pair, we try to merge the two clusters and see how it improves the ARI with the other methods. We then merge the pair that improves the ARI the most. We stop when the ARI cannot be improved anymore.

In the general case, we perform the ARI merging without the allen labels and we then use it as a comparison. A quick overview of how the algorithm performs with the allen labels is seen at the end.

3 Reduction in the number of clusters

4 Improvement in ARI

5 Comparison with Allen subclass

5.1 After ARI merging

##          ARI
## sc3     0.48
## Monocle 0.52
## seurat  0.52

5.2 After a last consensus step

5.3 Stopping during the merging

We can see that the ARI may not be the best metric for that comparison. # Seurat ARI Param

LS0tCmF1dGhvcjogIkhlY3RvciBSb3V4IGRlIELDqXppZXV4IgpkYXRlOiAnYHIgZm9ybWF0KFN5cy50aW1lKCksICIlZCAlQiAsICVZIilgJwpvdXRwdXQ6CiAgaHRtbF9kb2N1bWVudDoKICAgIHRvYzogdHJ1ZQogICAgdG9jX2RlcHRoOiAyCiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgIGNvZGVfZG93bmxvYWQ6IFRSVUUKICAgIApwYXJhbXM6CiAgZGF0YXNldDogU01BUlRlcl9udWNsZWlfTU9wIAogIHRpdGxlOiAiQW5hbHlzaXMgb2YgdGhlIFNNQVJUZXJfbnVjbGVpX01PcCBkYXRhc2V0IgotLS0KLS0tCnRpdGxlOiBgciBwYXJhbXMkdGl0bGVgCi0tLQoKYGBge3IgbG9hZCBwYWNrYWdlcywgaW5jbHVkZT1GfQpsaWJyYXJ5KGtuaXRyKQpvcHRzX2NodW5rJHNldCgKICBmaWcucG9zID0gIiFoIiwgb3V0LmV4dHJhID0gIiIsIHdhcm5pbmcgPSBGLCBtZXNzYWdlID0gRiwKICBmaWcud2lkdGggPSA1LCBmaWcuYWxpZ24gPSAiY2VudGVyIiwgZWNobyA9IEYKKQpsaWJzIDwtIGMoImhlcmUiLCAiZHBseXIiLCAiZ2dwbG90MiIsICJ0aWR5ciIsICJzdHJpbmdyIiwgInJlYWRyIiwgImNvd3Bsb3QiLAogICAgICAgICAgImNsdXN0ZXJFeHBlcmltZW50IiwgIm1jbHVzdCIsICJSQ29sb3JCcmV3ZXIiLCAicHJvZ3Jlc3MiLCAibWVyZ2VyIiwKICAgICAgICAgICJwbmciKQpzdXBwcmVzc01lc3NhZ2VzKAogIHN1cHByZXNzV2FybmluZ3Moc2FwcGx5KGxpYnMsIHJlcXVpcmUsIGNoYXJhY3Rlci5vbmx5ID0gVFJVRSkpCikKcm0obGlicykKdHlwZSA8LSBmdW5jdGlvbihkYXRhc2V0KSB7CiAgaWYgKHN0cl9kZXRlY3QoZGF0YXNldCwgIlNNQVJUIikpIHJldHVybigiU21hcnQtU2VxIikKICBpZiAoc3RyX2RldGVjdChkYXRhc2V0LCAiMTB4IikpIHJldHVybigiMTBYIikKICBzdG9wKCJUeXBlIHVua25vd24iKQp9CnR5cGUgPC0gdHlwZShwYXJhbXMkZGF0YXNldCkKbWVyZ2Vyc19ub19hbGxlbiA8LSByZWFkUkRTKGhlcmUoImRhdGEiLCB0eXBlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBwYXN0ZTAocGFyYW1zJGRhdGFzZXQsICJfbm9fYWxsZW5fbWVyZ2Vycy5yZHMiKSkpCmBgYAoKIyBFREEKCldlIGNhbiBmaXJzdCB2aXN1YWxpemUgdGhlIGRhdGEgYWZ0ZXIgZGltZW5zaW9uIHJlZHVjdGlvbiB3aXRoIHppbmJXYXZlLiBXZSB0aGVuIHVzZSB0LVNORSBhbmQgY29sb3IgdGhlIGNlbGxzIHdpdGggdGhlIGFsbGVuIGxhYmVscy4gU2luY2Ugd2UgdXNlIHZhcmlvdXMgdmFsdWVzIGZvciB0aGUgayBwYXJhbWV0ZXIsIHdlIHBsb3QgdHdvIHJlcHJlc2VudGF0aW9ucyB3aXRoIHRoZSBtb3N0IGV4dHJlbWUgdmFsdWVzIG9mIGsuCgpgYGB7ciB0LVNORSBwbG90cywgZmlnLndpZHRoPTE4LCBmaWcuaGVpZ2h0PTl9CnBsb3RzIDwtIGxpc3QuZmlsZXMoaGVyZSgiRmlndXJlcyIsICJFREEiKSkKcGxvdHMgPC0gcGxvdHNbc3RyX2RldGVjdChwbG90cywgcGFyYW1zJGRhdGFzZXQpXQpwbG90Lm5ldygpCnBsb3Qud2luZG93KDA6MSwgMDoxKQpyYXN0ZXJJbWFnZShyZWFkUE5HKGhlcmUoIkZpZ3VyZXMiLCAiRURBIiwgcGxvdHNbMV0pKSwgMCwgMCwgLjUsIDEpCnJhc3RlckltYWdlKHJlYWRQTkcoaGVyZSgiRmlndXJlcyIsICJFREEiLCBwbG90c1tsZW5ndGgocGxvdHMpXSkpLCAwLjUsIDAsIDEsIDEpCmBgYAoKCiMgR2VuZXJhbCBjb21tZW50cwoKV2UgaGF2ZSB0aGUgZm9sbG93aW5nIHdvcmtmbG93CgpgYGB7ciB3b3JrZmxvdywgZmlnLmhlaWdodD02LCBmaWcud2lkdGg9MTB9CnBsb3QubmV3KCkKcGxvdC53aW5kb3coMDoxLCAwOjEpCnJhc3RlckltYWdlKHJlYWRQTkcoaGVyZSgiRmlndXJlcyIsICJTTUFSVC1TZXFfd29ya2Zsb3cucG5nIikpLCAwLCAwLCAxLCAxKQpgYGAKCkluIHRoZSBTZXVyYXQgY2x1c3RlcmluZywgdGhlcmUgYXJlIHR3byBwYXJhbWV0ZXJzIHRvIGNob29zZSBmcm9tOiB0aGUgcmVzb2x1dGlvbiBhbmQgdGhlIGsucGFyYW0gKHVzZSBpbiBhIGtubiBzdGVwKS4gRnJvbSB0aGUgc2V1cmF0IGhlbHAgZmlsZSwgZm9yIHRoZSByZXNvbHV0aW9uIHBhcmFtZXRlcjogdXNlIGEgdmFsdWUgYWJvdmUgKGJlbG93KSAxLjAgaWYgeW91IHdhbnQgdG8gb2J0YWluIGEgbGFyZ2VyIChzbWFsbGVyKSBudW1iZXIgb2YgY29tbXVuaXRpZXMuCgpXZSBwaWNrZWQgYSB2YWx1ZSBvZiAxLjYgYXMgd2Ugd2FudCBtYW55IGNsdXN0ZXJzIHRvIHN0YXJ0IHdpdGguIFdlIGFsc28gcGljayBrID0gNTAuIFRob3NlIHZhbHVlcyBzZWVtIGludGVybWVkaWF0ZSBpbiB0ZXJtIG9mIEFSSSB3aXRoIG90aGVyIHBhaXJzIG9mIHBhcmFtdGVycyAoc2VlIHNlY3Rpb24gU2V1cmF0IEFSSSBwYXJhbSkKCldlIGFsc28gY29uc2lkZXIgdHdvIHdheXMgb2YgY29tcHV0aW5nIHRoZSBBUkkgYmV0d2VlbiBSU0VDIGFuZCBvdGhlciBtZXRob2RzLiBFaXRoZXIsIHdlIGZvcmNlIFJTRUMgdG8gYXNzaWduIGFsbCBjZWxscyB0byBhIGdpdmVuIGNsdXN0ZXIsIG9yIHdlIG9ubHkgY29tcHV0ZSB0aGUgQVJJIGJldHdlZW4gUlNFQyBhbmQgdGhlIG90aGVyIG1ldGhvZHMgb24gdGhvc2UgY2VsbHMgdGhhdCBSU0VDIGRvIGNsdXN0ZXIuIE5vdGUgdGhhdCBpbiB0aGUgc2Vjb25kIGNhc2UsIG90aGVyIHBhaXJzIG9mIG1ldGhvZHMgYXJlIGNvbXBhcmVkIHVzaW5nIGFsbCBjZWxscy4gV2UgZGVub3RlIGFzIFJzZWNUIHRoZSBjbHVzdGVyIGFzc2lnbmVtZW50IHdoZXJlIGFsbCBjZWxscyBhcmUgYXNzaWduZWQgKGkuZSBSc2VjIFRvdGFsKS4gCgpUaGUgQVJJIG1lcmdpbmcgbWV0aG9kIHdvcmtzIGFzIGZvbGxvdy4gV2UgaXRlcmF0ZSBvdmVyIGFsbCBwYWlycyBvZiBjbHVzdGVycyBmb3IgZXZlcnkgY2x1c3RlcmluZyBtZXRob2QuIEZvciBlYWNoIHBhaXIsIHdlIHRyeSB0byBtZXJnZSB0aGUgdHdvIGNsdXN0ZXJzIGFuZCBzZWUgaG93IGl0IGltcHJvdmVzIHRoZSBBUkkgd2l0aCB0aGUgb3RoZXIgbWV0aG9kcy4gV2UgdGhlbiBtZXJnZSB0aGUgcGFpciB0aGF0IGltcHJvdmVzIHRoZSBBUkkgdGhlIG1vc3QuIFdlIHN0b3Agd2hlbiB0aGUgQVJJIGNhbm5vdCBiZSBpbXByb3ZlZCBhbnltb3JlLgoKSW4gdGhlIGdlbmVyYWwgY2FzZSwgd2UgcGVyZm9ybSB0aGUgQVJJIG1lcmdpbmcgd2l0aG91dCB0aGUgYWxsZW4gbGFiZWxzIGFuZCB3ZSB0aGVuIHVzZSBpdCBhcyBhIGNvbXBhcmlzb24uIEEgcXVpY2sgb3ZlcnZpZXcgb2YgaG93IHRoZSBhbGdvcml0aG0gcGVyZm9ybXMgd2l0aCB0aGUgYWxsZW4gbGFiZWxzIGlzIHNlZW4gYXQgdGhlIGVuZC4KCiMgUmVkdWN0aW9uIGluIHRoZSBudW1iZXIgb2YgY2x1c3RlcnMKCmBgYHtyIEltcCBubyBhbGxlbn0KcGxvdFByZVBvc3QobWVyZ2Vyc19ub19hbGxlbikKYGBgCgojIEltcHJvdmVtZW50IGluIEFSSQoKYGBge3IgQVJJIGltcCBubyBhbGxlbiBjZWxsfQpwbG90QVJJUmVkdWNlKG1lcmdlcnNfbm9fYWxsZW4pCmBgYAoKCmBgYHtyIEFSSSB0cmVuZCwgZmlnLndpZHRoPTl9CkFSSXRyZW5kKG1lcmdlciA9IG1lcmdlcnNfbm9fYWxsZW4pCmBgYAoKIyBDb21wYXJpc29uIHdpdGggQWxsZW4gc3ViY2xhc3MKCiMjIEFmdGVyIEFSSSBtZXJnaW5nCgpgYGB7ciBjb21wIGNlbGx9CmFsbGVuX2NsdXN0ZXJzIDwtIHJlYWQuY3N2KGhlcmUoImRhdGEiLCB0eXBlLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHBhc3RlMChwYXJhbXMkZGF0YXNldCwgIl9jbHVzdGVyLm1lbWJlcnNoaXAuY3N2IikpLAogICAgICAgICAgICAgICAgICAgICAgICAgICBjb2wubmFtZXMgPSBjKCJjZWxscyIsICJjbHVzdGVyX2lkIikpCmNsdXN0ZXJzIDwtIHJlYWQuY3N2KGhlcmUoImRhdGEiLCB0eXBlLAogICAgICAgICAgICAgICAgICAgICAgICAgIHBhc3RlMChwYXJhbXMkZGF0YXNldCwgIl9jbHVzdGVyLmFubm90YXRpb24uY3N2IikpLAogICAgICAgICAgICAgICAgICAgICBoZWFkZXIgPSBUKQphbGxlbl9jbHVzdGVycyA8LSBmdWxsX2pvaW4oYWxsZW5fY2x1c3RlcnMsIGNsdXN0ZXJzKQpybShjbHVzdGVycykKCmFwcGx5KG1lcmdlcnNfbm9fYWxsZW4kY3VycmVudE1hdCwgMiwgZnVuY3Rpb24oeCkgewogICAgaW5kcyA8LSB4ICE9IC0xCiAgICB4YSA8LSB4W2luZHNdCiAgICB5IDwtIGFsbGVuX2NsdXN0ZXJzJHN1YmNsYXNzX2xhYmVsW2luZHNdCiAgICBtY2x1c3Q6OmFkanVzdGVkUmFuZEluZGV4KHhhLCB5KQp9KSAlPiUgZGF0YS5mcmFtZShBUkkgPSByb3VuZCguLCAyKSkgJT4lCiAgc2VsZWN0KEFSSSkKYGBgCgojIyBBZnRlciBhIGxhc3QgY29uc2Vuc3VzIHN0ZXAKCmBgYHtyIGZpbmFsIGNvbnNlbnN1c30KY2x1c3RlcnMgPC0gbWVyZ2Vyc19ub19hbGxlbiRjdXJyZW50TWF0CnIgPC0gd2hpY2goY29sbmFtZXMoY2x1c3RlcnMpID09ICJSc2VjVCIpCmlmIChhbGwuZXF1YWwoaW50ZWdlcigwKSAscikgIT0gVFJVRSkgewogIGNsdXN0ZXJzWywiUnNlYyJdIDwtIGFzc2lnblJzZWMobWVyZ2Vyc19ub19hbGxlbikgCn0KY2x1c3RlcnMgPC0gYXMubWF0cml4KGNsdXN0ZXJzKSAKCmNlbGxzQ29uc2Vuc3VzIDwtIENvbnNlbnN1cyhjbHVzTWF0ID0gY2x1c3RlcnMsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBsYXJnZSA9ICh0eXBlICE9ICJTbWFydC1TZXEiKSkKY29uc2Vuc3VzIDwtIGNiaW5kKGNlbGxzQ29uc2Vuc3VzLAogICAgICAgICAgICAgICAgICAgYWxsZW5fY2x1c3RlcnMkc3ViY2xhc3NfbGFiZWwsCiAgICAgICAgICAgICAgICAgICBjbHVzdGVycykKCmNvbG5hbWVzKGNvbnNlbnN1cylbYygxLCAyKV0gPC0gYygiQ29uc2Vuc3VzIiwgIkFsbGVuXG5zdWJjbGFzcyIpCnBsb3RDbHVzdGVycyhvYmplY3QgPSBjb25zZW5zdXMpCnRpdGxlKCJDb25zZW5zdXMgTWVyZ2luZyB3aXRoIHByb3BvcnRpb25cbmFuZCBjb21wYXJpc29uIHdpdGggdGhlIGFsbGVuIHN1YmNsYXNzLiIpCgpybShjbHVzdGVycywgY2VsbHNDb25zZW5zdXMsIGNvbnNlbnN1cykKYGBgCgojIyBTdG9wcGluZyBkdXJpbmcgdGhlIG1lcmdpbmcKCmBgYHtyIGNvbnNlbnN1cyB3aXRoIGludGVybWVkaWF0ZX0KbWlkTWF0IDwtIGludGVybWVkaWF0ZU1hdChtZXJnZXIgPSBtZXJnZXJzX25vX2FsbGVuLAogICAgICAgICAgICAgICAgICAgICAgICAgIHAgPSAuOSkKciA8LSB3aGljaChjb2xuYW1lcyhtaWRNYXQpID09ICJSc2VjVCIpCmlmIChhbGwuZXF1YWwoaW50ZWdlcigwKSAscikgIT0gVFJVRSkgewogIG1pZE1hdFssIlJzZWMiXSA8LSBhc3NpZ25Sc2VjKG1lcmdlcnNfbm9fYWxsZW4sIHAgPSAuOSkKfQptaWRNYXQgPC0gYXMubWF0cml4KG1pZE1hdCkKCmNlbGxzQ29uc2Vuc3VzIDwtIENvbnNlbnN1cyhjbHVzTWF0ID0gbWlkTWF0LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgbGFyZ2UgPSAodHlwZSAhPSAiU21hcnQtU2VxIikpCmNvbnNlbnN1cyA8LSBjYmluZChjZWxsc0NvbnNlbnN1cywKICAgICAgICAgICAgICAgICAgIGFsbGVuX2NsdXN0ZXJzJHN1YmNsYXNzX2xhYmVsLAogICAgICAgICAgICAgICAgICAgbWlkTWF0KSAlPiUgYXMuZGF0YS5mcmFtZSgpCgpjb2xuYW1lcyhjb25zZW5zdXMpW2MoMSwgMildIDwtIGMoIkNvbnNlbnN1cyIsICJBbGxlblxuc3ViY2xhc3MiKQpwbG90Q2x1c3RlcnMob2JqZWN0ID0gY29uc2Vuc3VzICU+JSBhcy5tYXRyaXgoKSkKdGl0bGUoIkNvbnNlbnN1cyBNZXJnaW5nIHdoZW4gc3RvcHBpbmcgZWFybHkiKQoKcm0oY2VsbHNDb25zZW5zdXMsIGNvbnNlbnN1cywgbWlkTWF0KQpgYGAKCmBgYHtyIHBsb3QsIGV2YWwgPSBGQUxTRX0KSW1wQVJJIDwtIEFSSXRyZW5kQWxsZW4obWVyZ2VyID0gbWVyZ2Vyc19ub19hbGxlbiwKICAgICAgICAgICAgICAgICAgICAgICAgYWxsZW4xID0gYWxsZW5fY2x1c3RlcnMkY2x1c3Rlcl9sYWJlbCwKICAgICAgICAgICAgICAgICAgICAgICAgYWxsZW4yID0gYWxsZW5fY2x1c3RlcnMkc3ViY2xhc3NfbGFiZWwsCiAgICAgICAgICAgICAgICAgICAgICAgIHZlcmJvc2UgPSBUUlVFKQoKZ2dwbG90KGRhdGEuZnJhbWUoc3RlcCA9IDA6bnJvdyhtZXJnZXJzX25vX2FsbGVuJG1lcmdlcyksCiAgICAgICAgICAgICAgICAgIHN1YmNsYXNzQWxsZW4gPSBJbXBBUklbLCAyXSwKICAgICAgICAgICAgICAgICAgY2x1c3RlckFsbGVuID0gSW1wQVJJWywgMV0pICU+JQogICAgICAgICBnYXRoZXIoa2V5ID0gInR5cGUiLCB2YWx1ZSA9ICJJbXBBUkkiLCAtc3RlcCksCiAgICAgICBhZXMoeCA9IHN0ZXAsIHkgPSBJbXBBUkkpKSArCiAgZ2VvbV9saW5lKGFlcyhjb2wgPSB0eXBlKSkgKwogIHNjYWxlX3lfY29udGludW91cyhsaW1pdHMgPSBjKDAsIDEpKQpgYGAKCldlIGNhbiBzZWUgdGhhdCB0aGUgQVJJIG1heSBub3QgYmUgdGhlIGJlc3QgbWV0cmljIGZvciB0aGF0IGNvbXBhcmlzb24uCiMgU2V1cmF0IEFSSSBQYXJhbQoKYGBge3Igc2V1cmF0IHBhcmFtfQpwbG90Lm5ldygpCnBsb3Qud2luZG93KDA6MSwgMDoxKQpyYXN0ZXJJbWFnZShyZWFkUE5HKAogIGhlcmUoIkZpZ3VyZXMiLCB0eXBlLCBwYXN0ZTAocGFyYW1zJGRhdGFzZXQsICJfc2V1cmF0X0FSSS5wbmciKSkpLAogIDAsIDAsIDEsIDEpCmBgYAo=